1 Issues in Empirical Studies with Students

Empirical studies in software engineering are often used to gain insight into new techniques and methods. However, due mainly to concerns of external validity, questions have been raised as to the value of these types of studies. In this paper, we discuss lessons learned from conducting a large number of empirical studies in university courses in three different countries (Italy, Norway and the United States). We provide a review of the relevant literature from the empirical software engineering community, the software engineering education community and the general education community. After identifying four important stakeholders in these studies (Researchers, Students, Instructors, and Professionals), we list the main goals and risks faced by each. We then use this information to develop a checklist that can be followed by researchers and educators when planning and running studies in university courses. The goal of this checklist is to provide guidance to ensure that such studies have as much experimental and pedagogical value as possible.